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ABSTRACT 



Through the U.S. Department of Education's Office of 



Educational Research and Improvement (OERI) , the government seeks to develop 
quality teaching techniques and encourage school districts across the country 
to adopt the most successful of them. However, in light of OERI ' s potential 
to contribute significantly to the quality of the nation's education, a 
review of its activities is especially disappointing. Its work is fragmented 
and apparently vulnerable to politicization and manipulation. This is 
especially true for OERI ' s regional education laboratories. By nearly all 
accounts, what they have produced has been largely irrelevant to the 
classroom. Their work suffers from research overlap, an overemphasis on 
service delivery at the expense of quality, and a lack of objectivity and 
responsible scientific methodology. Moreover, recent reports by the 
Department of Education's own inspector general have revealed significant 
mismanagement of federal funds at some laboratories. This paper suggests that 
laboratory funding should be tied to programs that boost academic outcomes. 
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Why Congress Should Overhaul 
The Federal Regional Education Laboratories 

Nina H. Shokraii^ 



As a shocking number of American children 
continue to perform poorly in the core subjects of 
mathematics and English, most Americans would 
be surprised to learn that their tax dollars are 
funding education frills developed by regional lab- 
oratories of the U.S. Department of Education. 
According to Diane Ravitch and Chester Finn, 
former Assistant Secretaries at the Department of 
Education's Office of Educational Research and 
Improvement (OERI), which oversees these labo- 
ratories, regional education labs are nothing more 
than 



A collection — now ten — of smallish non- 
profit organizations, each located in a dif- 
ferent region, that undertake a mishmash 
of research, dissemination, and technical 
assistance activities, aimed mostly at state 
and local education agencies.... The pro- 
gram as a whole has outlived whatever 
justification it once had.... The putative 
beneficiaries are schoolchildren who need 
to learn more. Yet the money actually goes 
to well-paid professionals functioning as 
middlemen, sitting in comfortable offices 
distant from the classroom, and devoting " 



much of their energy to ensuring that their 
federal gravy train does not halt on the 
tracks.^ 



The federal government 
can play an important role in 
gathering information, con- 
ducting research, and dis- 
seminating valid findings. 
There may not be a “magic 
bullet" for every problem in 
Americas failing schools, but 
educators have developed 
many effective teaching 
methods that could be repli- 
cated in classrooms across 
the country. Currently, OERI 
is in charge of this function. 
Its National Center for Edu- 
cation Statistics (NCES) has 
gathered, analyzed, and dis- 
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seminated national data. But the remainder of 



OERIs work is fragmented and highly vulnerable 
to politicization and manipulation. 



This is especially true with respect to OERIs 
regional education laboratories. These labs have 



1 . The author would like to thank Jake Phillips, a 1998 summer intern at The Heritage Foundation from Duke University, for 
his contributions to this study. 

2. Diane Ravitch and Chester Finn, Education Reform, 1995-1996 (Indianapolis, Ind.: Hudson Institute, 1996). 
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been around for more than 32 years and have 
received over $750 million in federal funding. Yet, 
by nearly all accounts, what they produce has had 
little impact in revolutionizing education in Amer- 
icas schools. 

To a great extent, reformers have been frustrated 
in their efforts to improve the labs’ effectiveness: 
The influence of lobbyists who promote fanciful 
education projects has been too strong — and the 
resolve of Congress has been too weak — for mana- 
gerial and substantive change to be implemented 
successfully. 

Education research is best conducted by private 
and independent organizations that are funded 
based on their ability to translate research findings 
into successful methodology and practice. The 
sole determinant in deciding whether a program 
will be implemented in the schools should be an 
increase in academic scores, not how much the 
educational bureaucracy or lobbyists like the pro- 
gram. Yet the laboratories and the federal bureau- 
cracy rarely analyze how well programs are 
working. 

Members of Congress interested in streamlining 
the federal role in education should examine the 
research conducted in OERI laboratories and 
demand more accountability in return for the 
approximately $50 million the labs receive from 
the federal government every year. The best way to 
achieve this goal would be to allow the private sec- 
tor, as well as other agencies, to compete for funds 
targeted to the work of these labs, and to base this 
funding on individual projects. Or Congress could 
simply roll over the labs’ funding into a block 
grant to states, which then could set their own 
research priorities and hire whomever they see fit 
to conduct the work. 



WHAT FEDERAL EDUCATION LABS DO 

The largest portion of the federal government’s 
role in education research is administered through 
four programs of the Office of Educational 
Research and Improvement, at a total cost of $510 
million in fiscal year (FY) 1998. These programs 
include: 

1 . Regional Educational Laboratories (10) ; 

2 . National Education Research Institutes (5) in 
charge of 1 1 National Research Centers; 

3 . Educational Resources Information Centers 
(ERIC) ; and 

4 . Eisenhower National Clearinghouse for Mathe- 
matics and Science, and Regional Consortia. 

Regional education laboratories were created in 
1965 as part of President Lyndon B. Johnson’s 
Great Society education initiative, the Elementary 
and Secondary Education Act (ESEA) . The goal of 
the labs, according to the U.S. Department of Edu- 
cation, is to bring “structure and cohesion to a 
fragmented federal role in educational research 
and development.”^ They are the Department’s 
largest research and development investment. In 
1996, Congress appropriated around $51 million 
for the labs, and this is supplemented by funds 
obtained from grants and contracts through the 
Department of Education and other federal, state, 
and local sources."^ 

Contracts to run the labs are awarded every five 
years. The labs report to regional boards com- 
prised primarily of educators and. in many cases, 
the chief state school officers.^ 

A Snapshot of the Regional Labs^ 

• The Appalachian Educational Laboratory 
(AEL) is based in Charleston, West Virginia, 



3. Nancy Kober, From Promise to Practice: Stories from the Regional Educational Laboratories, U.S. Department of Education, 
Office of Educational Research and Improvement, September 1996; see http://www.ed.gov/prog_info/Labs/Promise/over- 
view.html. 

4. Ibid. 

5. Interview with Robert Stonehill, Director of the State and Local Services Division of the Office of Reform Assistance and 
Dissemination. U.S. Department of Education. Office of Educational Research and Improvement, May 20, 1998. 
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and serves Kentucky, Tennessee, Virginia, and 
West Virginia. One of the oldest labs, it special- 
izes in rural education and houses the ERIC 
Clearinghouse on Rural Education and Small 
Schools. It promotes the integrity of rural small 
schools in a global economy AEL received 
$4.1 million in FY 1996. It developed the well- 
known Questioning and Understanding to 
Improve Learning and Thinking (QUILT) 
teaching model and has studied the Kentucky 
Education Reform Act of 1990 to analyze the 
effects of large-scale changes in state policy on 
rural education. It also publishes Family Con- 
nections, a weekly guide for parents used in 45 
states and available to more than 60,000 
families.^ 

• The Mid-Atlantic Laboratory for Student Suc- 
cess (LSS) is based in Philadelphia, Pennsylva- 
nia, and^serves Delaware, the District of 
Columbia, Maryland, New Jersey and Pennsyl- 
vania. It specializes in urban education and 
received $5.2 million in FY 1996. An Eisen- 
hower Regional Consortium is located in Phil- 
adelphia. LSS runs the Learning City Program, 
a broad school-family-community approach 
to promoting the educational attainment of 
urban youth. Studies show that students in the 
program achieve higher math and reading 
scores on standardized tests, as well as higher 
levels of self-esteem.^ 

• The Mid-Continent Regional Educational Lab- 
oratory (McREL) is based in Aurora, Colorado, 
and serves Colorado, Kansas, Missouri, 
Nebraska, North Dakota. South Dakota, and 
Wyoming. It specializes in curriculum, train- 
ing. and instruction and received $4.2 million 
in FY 1996. McREL has been heavily involved 



in developing national standards. It also has 
developed a system to modernize the South 
Dakota education system and has published 
The Systemic Identification and Articulation of 
Content Standards and Benchmarks.^ 

• The North Central Regional Educational Labo- 
ratory (NCREL) is based in Oak Brook, Illi- 
nois, and serves Iowa, Illinois, Indiana, 
Michigan, Minnesota, Ohio, and Wisconsin. It 
specializes in technology education and 
received $6.6 million in FY 1996. This lab has 
studied, among other things, the Iowa legisla- 
tures early childhood programs to help target 
state funding. It also has developed 22 pro- 
fessional development programs and has trans- 
mitted them to more than 23,000 schools 
across the region. 

• The Northeast and Islands Regional Educa- 
tional Laboratory (LAB) is located at Brown 
University in Providence, Rhode Island, and 
serves Connecticut, Maine, Massachusetts, 
New Hampshire, New York, Puerto Rico, 
Rhode Island, Vermont, and the Virgin Islands. 
It specializes in language and cultural diversity 
and received $6.1 million in FY 1996. An 
Eisenhower Regional Consortium is located in 
Cambridge, Massachusetts, at the Eisenhower 
Regional Alliance for Mathematics and Science 
Education Reform. LAB is one of the newest 
labs, having taken over another lab that was 
caught overbilling the government. It currently 
is exploring strategies to implement academic 
standards for English-language learners and to 
identify math and science instructional prac- 
tices that effectively address the needs of 
diverse learners. 



6. This section draws on a research document prepared by Josh Smith, an intern for the House Committee on Education and 
the Workforce, “Overview/Analysis of ED Regional Lab Program and Other OERI Research Programs,” July 7, 1997. 

7. See http://www.ael.org. 

8. See http://www.temple.edu/departments/LSS. 

9. See http://www.ed.gov/prog_info/Labs/Profiles/mcrel.html. 

10. Kober, From Promise to Practice. 

1 1. See http://wwwvlab.brown.edu. 
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• The Northwest Regional Educational Labora- 
tory (NWREL) is based in Portland, Oregon, 
and serves Alaska, Idaho, Montana, Oregon, 
and Washington State. It specializes in school 
processes and received $5.2 million in FY 
1996. An Eisenhower Regional Consortium is 
located in Portland as well. NWREL has devel- 
oped Onward to Excellence, a school reform 
model used by 2,000 schools over the past 14 
years, and a program on teaching students 
how to write well. It also has trained thou- 
sands of teachers and assists schools techni- 
cally. 

• The Pacific Region Educational Laboratory 
(PREL) is based in Honolulu. Hawaii, and 
serves American Samoa. Guam. Hawaii, the 
northern Marianas Islands, the Marshall 
Islands. Micronesia, and Palau. It specializes in 
language and cultural diversity and received 
$3 million in FY 1996. PRELs major function is 
the training and networking of remote island 
teachers. It also attempts to integrate island 
culture into local education and holds the 
Pacific Education Conference, which draws 
about 1.200 participants each year. PREL has 
studied school finance, at-risk students, and 
cultural learning in the Pacific. 

• The Southern Regional Vision for Education 
(SERVE) is based in Greensboro. North Caro- 
lina. and serves Alabama. Florida. Georgia, 
Mississippi. North Carolina, and South Caro- 
lina. It specializes in early childhood education 
and received $5.6 million in FY 1996. SERVE 
studies site-based management in North Caro- 
lina and dropout prevention in South Carolina. 
It also has helped develop a school board 
member training program in Mississippi and 
has conducted a study of school restructuring. 



• The Southwest Educational Development Lab- 
oratory (SEDL) is based in Austin. Texas, and 
serves Arkansas. Louisiana. New Mexico. 
Oklahoma, and Texas. Specializing in language 
and cultural diversity, it received $5.5 million 
in FY 1996. SEDL monitors a program to fos- 
ter understanding and plan for the education 
of students on both sides of the border. 

• The WestEd Laboratory (WestEd) is based in 
San Francisco. California, and serves Arizona, 
California, Nevada, and Utah. It specializes in 
assessment and accountability. WestEd 
received $5.5 million in FY 1996. Its principal 
research effort has been a study of situational 
teaching, a model now used by more than 
4,800 teachers in California. The lab also 
engages in "whole school" reform, language 
and cultural diversity, state alliance projects, 
and policy support and studies programs. In 
addition to developing the Infant/Toddler Car- 
egiver Training Program, it has adapted the 
Success for All reading program from the 
National Research Center at Johns Hopkins 
University to serve students with limited 
English proficiency more effectively. WestEd 
has helped more than 100 schools adopt and 
implement the Success for All model, which is 
touted by OERI as one of the top 16 lab 
projects, to meet local needs. 

WHArS WRONG 

WITH FEDERAL EDUCATION LABS 

Although the ten regional laboratories seem to 
be doing interesting work, their impact on educa- 
tion reform has been small; in some cases, their 
work may have been undermined efforts at reform. 
Among the many problems associated with the 
work of these labs are the following: 



12. See http://www.ed.gov/prog_info/Labs/ProfHes/nwrel.html. 

13. Kober, From Promise to Practice. 

14. See http://www.ed.gov/prog_info/Labs/proriles/prel.htmI. 

15. See http://www.ed.gov/prog_info/Labs/profiles/sedl.html. 

16. Kober. From Promise to Practice. 
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1. Evidence of Waste and Abuse. A March 1998 
audit of WestEd by the U.S. Department of 
Educations own inspector general (IG) found 
that the lab did not comply with certain federal 
laws and regulations in managing its Regional 
Educational Lab contracts. The audit also 
found that WestEds indirect cost estimates 
negotiated by the Department of Education do 
not reflect all of its indirect costs. Specifi- 
cally: 

• WestEd leased space to a radio station and 
a computer facility in buildings purchased 
with federal funds for educational research 
purposes and retained all subsequent prof- 
its, totaling $627,000, from December 
1994 to November 1996. 

• WestEd used "lease-purchase agreements 
that resulted in excessive charges for furni- 
ture, equipment, and building improve- 
ments. The interest portion of the lease- 
purchase payments was excessive since 
WestEd had funds available in its reserve to 
make cash purchases. WestEd also acceler- 
ated charges to the contract for the pur- 
chases and charged interest to the contract 
during periods when interest was an unal- 
lowable cost." 

• WestEd “improperly billed the contract for 
indirect costs on work performed by sub- 
contractors.” 

• It also “gave the impression that indirect 
costs remained fairly constant when in fact 
the indirect cost rate increased 29 percent 
over the past three years. For fiscal year 
1996, WestEds actual indirect cost rate 
was 45 percent rather than its stated rate of 
12.8 percent.” 



The inspector general recommended that the 

department require WestEd to use federal 

funds and property efficiently and for their 

intended purposes. The IG also required 

WestEd to return approximately $131,000 in 

federal funds used for unallowable interest, 

improperly computed indirect costs, and other 

unallowable direct costs. In addition, the IG 

concluded that WestEd should use the $2.6 

million of accumulated rental profits to reduce 

program objectives and expenses, leaving over 

$300,000 in rental profits available for these 

1 8 

purposes annually in future years. 

Another audit, on February 25, 1998, dealt 
with the costs incurred by the University of 
North Carolina, Greensboro, and the South- 
eastern Regional Vision for Education under 
the U.S. Department of Educations Regional 
Educational Laboratories contract. This 
audit found that the university “could not pro- 
vide assurance that approximately $2.3 million 
in salaries, fringe benefits, and related indirect 
costs between December 1, 1995 and Novem- 
ber 30, 1996 were allocated in direct relation- 
ship to the time and effort made toward the 
Regional Educational Laboratories contract." 
The department also was unable to “make an 
informed decision regarding $415,000 of pro- 
posed subcontract actions because the Univer- 
sity did not obtain cost information from 
prospective subcontractors as required by the 
Federal Acquisition Regulation." Though the 
university has taken action to correct itself, the 
inspector general recommended that the 
department require the university to submit 
additional information for evaluation and 
assurance of compliance with appropriate fed- 
eral regulations. 



17. U.S. Department of Education, Office of the Inspector General, "WestEds Administration of the Regional Educational Lab- 
oratory Contracts,” March 31. 1998. 

18. Ibid. 

19. U.S. Department of Education, Office of the Inspector General. "Review of Costs Incurred by the University of North Caro- 
lina, Greensboro, and the Southeastern Regional Vision for Education Under the U.S. Department of Educations Regional 
Educational Laboratories Contract,” February 25, 1998. 
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2. Questionable Quality and Value of Research. 
As Peirce Hammond, director of the Office of 
Reform Assistance and Dissemination at OERI, 
explains, labs have to “strike a balance 
between the urgency of children’s needs. The 
issue is what do you do with partial knowl- 
edge. Take a chance? Or take a risk? Labs are 

„po 

in the middle of this dilemma all the time. 

But a 1997 District and State Survey con- 
ducted by the U.S. Department of Education 
found that nearly half of the states and districts 
that had contact with regional labs found them 
to be of little or no help in understanding or 
implementing comprehensive standards-based 
reform.^ ^ Their emphasis on speedy delivery is 
also not reassuring, because sound research 
methods often rely on experiments that take 
months, if not years, to conduct. 

Lack of Objectivity. The labs consider them- 
selves to be independent and objective and, 
through their national network, to possess a 
capacity to tackle nationwide needs and 
encourage efficiency. “By using proven prac- 
tices and applying Laboratory expertise, edu- 
cators and policymakers can avoid 
duplications, mistakes, and unnecessary costs 
of reinventing programs,” boasts one lab man- 

ual.22 

But even a quick review of lab Internet sites 
shows big gaps in the information made avail- 
able. There are few details on free-market ideas 
or initiatives such as school choice, even 
though there are many recent studies on 
school choice programs and their positive 
appeal to inner-city parents (especially with 
the increase in privately sponsored programs) . 
Somewhat more information is available on 



charter schools and the contracting-out of 
school services. 

3. Promotion of Fads. The labs are supposed to 
disseminate proven methods of achieving aca- 
demic improvement, yet many of the methods 
they promote are widely criticized fads, such 
as whole language. Furthermore, the research 
they conduct often seems to rely more on 
anecdotes and theories than on facts. 

For example, OERI lists AELs Family Connec- 
tions as one of the Regional Laboratories’ top 
16 tested ideas for teaching and learning. Yet 
the program was reviewed by a panel selected 
by the AEL that included Head Start coordina- 
tors, Title I directors, teachers, and Parent- 
Teacher Association leaders. The panel gave 
high marks to the program for “developmental 
appropriateness, interest, understandability, 
and usefulness to parents," but failed to men- 
tion any impact on students’ academic 
achievement.^^ 

AEL also touts its QUILT program, which 
was designed to “train teachers to use well- 
placed questions to stimulate students to 
think, question, and learn." QUILT “helps 
teachers create a classroom environment that is 
more reflective, more student centered, more 
inquiry based, and more metacognitive.’’^^ Yet 
no one — not even the labs — can provide evi- 
dence that QUILT actually improves students’ 
test scores. The data from a four-state field test 
indicate only that QUILT-trained teachers 
employed what they learned to change class- 
room practice and that students in their class- 
rooms asked more questions that reflected 
“better thinking. 



20. Authors interview. May 20, 1998. 

21. U.S. Department of Education. Office of the Under Secretary, "Reports on Reform from the Field: District and State Survey 
Results: Final Report," prepared by the Urban Institute, Washington, D.C., 1997. 

22. Kober. From Promise to Practice. 

23. Luna Levinson and Robert Stonehill, “Tried and True: Tested Ideas for Teaching and Learning from the Regional Educa- 
tional Laboratories," U.S. Department of Education, Office of Educational Research and Improvement. September 1997. 

24. Robert M. Stonehill and Lynn M. Spencer. Profiles of the Regional Educational Laboratories. U.S. Department of Education. 
Office of Educational Research and Improvement, December 1997. 
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An LSS project entitled “The Rural Cluster 
Case Study: Say Yes to a Youngsters Future” 
promotes a math and science curriculum 
“designed to be inquiry-based, hands-on. 
cooperative, [and] self-paced.” There is, how- 
ever, no mention of whether the program is or 
is not backed by research or solid evidence 
that demonstrates proven academic gains. LSS 
claims that the overall goal of this program is 
to test the feasibility of implementing the “Say 
Yes” program and to determine how to scale 
up implementation — with no mention of see- 
ing how “Say Yes” improves academic achieve- 
ment.^^ 



The labs also frequently use terms like “feel- 
ing positive” about the classroom and school 
environment, “situational learning,” and “bet- 
ter academic self-concept” to describe what 
they are researching. NCREL developed “Stra- 
tegic Reading,” an “innovative” approach that 
uses reading to improve instruction and 
encourages “team teaching, active roles for stu- 
dents, and meaningful principal participation,” 
among other things. NWREL is developing 
processes and resources for schools to establish 
classroom environments that are developmen- 
tally and culturally appropriate for young chil- 
dren. 



THE LABS' QUESTIONABLE RESEARCH 

E. D. Hirsch, a professor at the University of 
Virginia, points out that terms such as “develop- 
mentally appropriate,” “modern,” “hands-on,” 
“integrated." “interesting,” and “individualized” are 
often used to describe teaching techniques that are 
ideologically loaded and neither grounded in nor 
backed by solid research that tests their effective- 



ness. Several studies show, for instance, that learn- 
ing ability does not correlate with age and 
background — an idea that underpins “develop- 
mentally appropriate” programs. Juxtaposing 
developmentally appropriate methods with tradi- 
tional methods of teaching students makes the 
“progressive” techniques more attractive, but little 
if any research supports their effectiveness.^^ 

Labs have different methods of evaluation and 
place different degrees of importance on research. 
Yet all use the term “research-based” to describe 
the programs they promote, even though "[they] 
don’t have the luxury of running controlled exper- 
iments,” as Wesley Hoover, President and CEO of 
the Southwest Educational Development Labora- 
tory, admits. At the same time, “Direct Instruc- 
tion,” a program which uses memorization and 
drill and has been proven to be effective in boost- 
ing the academic outcomes of disadvantaged stu- 
dents, is not widely distributed or prominently 
featured by the labs because it is not interesting to 
the teachers who have to use it. “[The] program 
will not work," claims Hoover, because “75 per- 
cent of teachers in our Elementary schools come 
from a whole language background. You need 
something else that ‘walks them through Direct 
Instruction.’” 

Flaws in Lab Research 

A June 1993 analysis of OERI and its labs by 
Maris Vinovskis, former Research Advisor to the 
Assistant Secretary at OERI and currently a profes- 
sor at the Department of History and Institute for 
Social Research at the University of Michigan, pro- 
vides a glimpse of the quality of research and 
development at five OERI Regional Education 



25. See http://www.ael. org/quilt.htm. 

26. Josh Smith, “Education Research — Models, Issues, Problems," report produced for the Committee on Education and the 
Workforce, U.S. House of Representatives, July 31, 1997. 

27. Ibid. 

28. E. D. Hirsch, Jr., The Schools We Need and Why We Don’t Have Them (New York: Doubleday, 1996). 

29. Interview, May 20, 1998. 

30. Bonnie Grossen, “Making Research Serve the Profession," American Educator. Fall 1996. See also Tyce Palmaffy, “No 
Excuses," Policy Review, January-February 1998. 
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labs. Vinovskis praised the work of some of the 
labs, but he also found several problems: 

The mission of the Northeast and Islands 
Regional Educational Laboratory (LAB) at Brown 
University is to “promote systemic school 
improvement in the Northeast and Islands region 
by supporting researchers who conduct collabora- 
tive inquiry alongside educational practitioners 
and community members.” Yet LAB launched an 
intensive Collaborative Action Research effort in 
three sites without an adequate research design for 
a large-scale or long-term undertaking.^^ LAB has 
put together a “nice series of briefing papers on 
teacher quality pension portability and teacher 
incentives,” but the product only summarizes 
existing policy papers. 

The main emphasis of the Southwest Educa- 
tional Development Laboratory (SEDL) is “ensur- 
ing educational equality for mostly minority or 
handicapped children and youth who live in pov- 
erty. One of SEDLs projects was an investigation 
of small and economically disadvantaged rural 
schools in five different states over several years. 
Each of the five demonstration schools, all of 
which had indicated an interest in improving stu- 
dent achievement scores, was visited monthly by 
an SEDL staff member responsible for facilitating 
change in that institution. An analysis of the plan 
found that because of a lack of standardized mea- 
sures of educational success (and failure to include 
adequate controls in the research design) , it was 
impossible to make any systematic comparisons 
among the schools. And since the schools studied 



were not necessarily typical of others in that 
region, it was difficult to know how much could 
be generalized from these findings. Because there 
were no control groups in the study, it was hard to 
establish which factors were instrumental in 
improving student achievement. The analysis 
found the SEDL project an interesting but very 
limited demonstration effort at five diverse sites, 
which does not lead to information that would be 
useful in determining key factors in improving 
rural education. 

The Mid-Continent Regional Educational Labo- 
ratory (McREL) emphasizes working “collabora- 
tively with its clients to improve educational 
policy and practice through the application of 
knowledge from research, development, and expe- 
rience.”^^ It has produced some useful policy 
papers, but many are little more than a catalog of 
research, without adequate attention given to the 
quality of the work or the diversity of material in 
the field. In addition, there are serious flaws in 
the way the lab tallied the results of a question- 
naire it sent to local public school superintendents 
in the seven states it serves. Although the overall 
response rate was only 40 percent and varied 
among the states, the analysts simply grouped all 
of the returns together to get an overall regional 
profile. 

The signature program of the Northwest 
Regional Educational Laboratory (NWREL) — 
“Onward to Excellence” (OTE) — is referred to as a 
“research-based” school improvement process that 
focuses the attention and resources of school sys- 



31. Maris A. Vinovskis, "Analysis of the Quality of Research and Development at the OERI Research and Development Centers 
and at the OERI Regional Educational Laboratories," Final Report, Office of the Assistant Secretary. U.S. Department of 
Education, Office of Educational Research and Improvement, June 1993. Vinovskis served as Research Advisor to the 
Assistant Secretary of OERI under Diane Ravitch (Bush Administration) and Sharon Robinson (Clinton Administration). 

32. Ibid. 

33. Ibid. 

34. Stonehill and Spencer. Profiles of the Regional Educational Laboratories, p. 24. 

35. Vinovskis, "Analysis,” pp. 169-170. 

36. Stonehill and Spencer, Profiles of the Regional Educational Laboratories, p. 28. 

37. Vinovskis. “Analysis," p. 167. 

38. Ibid. 
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terns and their communities on learning success 
for all students and develops the capacity within 
school systems for continuous improvement 
through shared leadership. OTE was tested on 
nearly 2,000 schools, yet a study of its impact did 
not explain why certain OTE programs were cho- 
sen for the study and others were not. Moreover, 
an in-depth study of OTE programs could not 
explain why some of them succeeded while others 
failed. The study failed to look at non-OTE 
schools to see whether any of the improvements in 
the OTE schools were due in part to other general 
changes affecting all schools during those years. 
And because the term “success" was not very well 
defined, OTE programs that set low goals conceiv- 
ably could be labeled as “successful" as — or even 
more “successful" than — those that set higher 
goals. 

Vinovskis concluded that after spending $811 
million of taxpayer money between 1966 and 
1991, the labs had little to show for it by way of 
success. 

FAILED EFFORTS AT REFORM 

The inefficiencies and contradictions of the 
regional labs have not escaped notice. In fact, part 
of a congressional reorganization of OERI in 1994 
specifically addressed these problems.^ ^ The reor- 
ganization included the creation of an Office of 
Assistance and Dissemination (ORAD), designed 
primarily to guide and monitor the activities of the 
regional educational laboratories. The labs were 
required to uphold the agreements in their five- 
year contracts with the government and to pro- 
mote the goals identified by their individual 
boards. They also were given two quantifiable 
tasks: (1) bring together scattered successful 
reform efforts at the state and local levels, and (2) 



place more emphasis on existing successful 
reforms to promote their implementation in other 
areas of the country. 

Thus, the 1994 reforms provided a basis for 
OERI action both to harness the activities of the 
labs and to promote greater accountability in and 
better guidance for their operations. Even Pre- 
ston Kronkosky, Executive Director of the South- 
west Educational Laboratory, observed that the 
new arrangement provided a valuable opportunity 
for the labs to streamline their goals and conform 
them to the legislation. 

This initial optimism was quickly dashed, how- 
ever, by the lobbying efforts of the Council on 
Educational Development and Research (CEDaR). 
Established initially to represent the interests of 
these labs, CEDaR apparently found the Republi- 
can-led 104th Congress as receptive as the Demo- 
cratic-controlled one before it. A major blow to the 
1994 reform came in 1996 when Congress, in its 
appropriations report, claimed that all of the labs’ 
work should be based on the directives of their 
regional governing boards. This directly opposed 
the stipulation tying them to their contracts with 
OERI that had been imposed only two years ear- 
lier. 

WHAT POLICYMAKERS SHOULD DO 

The best way to reform the regional education 
laboratories is to incorporate competition and 
sound research standards into the process. 

Labs should compete with other agencies and 
with the private sector for federal grants to fund 
each project. Moreover, given today’s widespread 
access to the Internet and the existence of various 
organizations that accumulate credible informa- 
tion, the education bureaucracy no longer can jus- 



39. Stonehill and Spencer, Profiles of the Regional Educational Laboratories, p. 37. 

40. Vinovskis. “Analysis," pp, 168-169, 

41. “Regional Educational Laboratories for Research, Development, Dissemination. Technical Assistance. RFP No. 95-040; 
Statement of Work,” U.S. Department of Education, Office of Educational Research and Improvement. 1995, p. 3. 

42. Maris A, Vinovskis, “Changing Federal Strategies for Supporting Educational Research, Development, and Statistics,” pre- 
liminary draft prepared for OERIs National Education Research Policy and Priorities Board. October 1997. 

43. Ibid. 
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tify wasting taxpayer dollars by charging regional 
laboratories with gathering information. 

The labs’ research methods and the activities 
they use to disseminate their conclusions must be 
overhauled. Policymakers should adopt a set of 
guidelines to govern decisions on how labs con- 
duct and disseminate research. Such guidelines 
should include: 

• Rescuing “research” from nihilism. One prob- 
lem with much of the labs’ research lies in 
“results” that cannot be applied to other cases 
because the subjects studied were not con- 
trolled for all variables. Jay P. Greene, an assis- 
tant professor of government at the University 
of Texas at Austin and a research associate at 
Harvard University’s Program on Education 
Policy and Governance, has suggested one way 
to improve the quality of research. He main- 
tains that educational researchers should use 
“random assignment” in selecting subjects for a 
study, much as the medical research commu- 
nity does. In controlled studies using random 
assignment, two groups are tracked that, on 
average, are exactly alike except for the type of 
“treatment" being tested. “Thus,” says Greene, 
“any differences observed between the ran- 
domly assigned groups after a period of time 
■ can be attributed to the treatment. SEDEs 
Hoover contends that “when you work with 
schools, you can’t afford to conduct random 



experiments.”^^ But the alternative is conduct- 
ing an experiment on students without a viable 
mechanism with which to measure the experi- 
ment’s success. 

• Raising research standards. As Bonnie Grossen 
illustrates in the Fall 1996 American Educator, 
many experimental practices today jumped 
from theory or hypothesis to the classroom 
without first being tested by formal experi- 
mentation or analysis to determine the validity 
of the underlying hypothesis. According to 
Arthur Ellis and Jeffrey Fouts, professors of 
education at Seattle Pacific University, there are 
three different levels of research. Level I is 
theory-building, in which the researcher devel- 
ops a hypothesis. Level II tests the theory 
through formal experimentation and analysis 
of data to determine the validity of the hypoth- 
esis. Level III replicates the results in large- 
scale studies and school- or district-wide 
implementation. Level III research calls for 
peer review and replication of the experiment, 
as well as large-scale or long-term follow-up 
studies. Unfortunately, many of the programs 
being touted by the regional labs have not had 
any Level III or even Level II research. Rob- 
ert Stonehill of OERI believes that the current 
role of the labs is to move research from Level I 
to Levels II and III.*^^ Though this is reassur- 
ing, few parents will find it comforting that 



44. Jay P Greene. “Rescuing Education Research,” Education Week, April 29, 1998. 

45. Authors interview. May 20. 1998. 

46. For example, jean Piagets work in developmental psychology is at Level I (basic). Piaget never tried to teach children; he 
only observed what children do at different stages. Project Follow-Through is an example of Level III research (the most 
advanced). From 1967 until the summer of 1995, this $1 billion project studied more than 70,000 students in more than 
180 schools. The projects goal was to find a specific teaching method that raised the academic performance of the poor. An 
independent agency evaluated the study. The model found "Direct Instruction,” a method that relies more on memoriza- 
tion and drill, the most successful at boosting academic achievement in poor children. Yet the National Institute of Educa- 
tion (NIE), the predecessor of OERI, discredited Project Follow-Through in 1981, saying that "the audience for Follow- 
Through evaluations is an audience of teachers that doesn't need statistical finding of experiments to decide how best to 
teach children. They decide such matters on the basis of complicated public and private understandings, beliefs, motives 
and wishes.” 

47. Arthur K. Ellis and Jeffrey T. Fouts, Research on Education Innovation (New York: Eye on Education, 1997). 

48. Grossen, "Making Research Serve the Profession." 

49. Authors interview, May 20, 1998. 
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education methods based only on Level I 
research are being used with their children. 

• Defining what “research based” and other 
terms mean. At the heart of the problem are 
the vague definitions and misuse of federal 
education “research” terms and the phrase 
“research based.” Robert Sweet, former head of 
the National Institute of Education (NIE), 
OERIs predecessor agency has been an out- 
spoken advocate of redefining research terms. 
For example, he believes such terms as “exem- 
plary” and “promising” should be replaced 
with stricter terms such as “reliable and repli- 
cable” and “pilot tested” — assuming that they 
actually apply The term “reliable replicable 
research” should mean only objective and valid 
scientific studies that (1) include rigorously 
defined samples of subjects that are sufficiently 
large and representative to support the general 
conclusions; (2) rely on measurements that 
meet established standards of reliability and 
validity: (3) test competing theories where 
multiple theories exist; (4) are subjected to 
peer review before their results are published; 
and (5) discover effective strategies for improv- 
injg academic skills. The ultimate test of a pro- 
grams usefulness should be whether the 
students in the program excel academically. 
The federally funded labs should replicate only 



those methods and education plans that have 
been proven to boost academic achievement. 

CONCLUSION 

Members of Congress can take a grand step to 
promote sound education that achieves success in 
Americas classrooms. The solution lies in the 
nations commitment to proven models of educa- 
tion success. 

Federal education research labs, as currently 
funded and operated, have done little to promote 
these successful models. Their funding should be 
tied to programs that boost academic outcomes. 
The regional labs also should compete with other 
agencies and the private sector for federal grants to 
fund each individual project, or else their func- 
tions should be rolled over to the states to enable 
each state to set up its own research entities. 

Finally policymakers should establish simple 
guidelines to govern decisions on how to conduct 
and disseminate research. The sole determining 
factor in deciding whether a program will be 
funded and implemented in the schools should be 
an observed increase in academic scores, not how 
much the educational bureaucracy or education 
lobbyists like the program. 

— Nina H. Shokraii is Education Policy Analyst at 
The Heritage Foundation. 
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Why Congress Should Overhaul 
The Federal Regional Education Laboratories 

Nina H. Shokraii 



As Congress and the President search for ways 
to send more dollars to the classroom, cut waste ‘ 
and bureaucracy and assure Americans that all 
children^ — regardless of their background and 
income — receive a quality education, the federal ' 
governments role in researching and disseminat- 
ing proven teaching methods should naturally 
attract their attention. 

Through the U.S. Department of Educations 
Office of Educational Research and Improvement 
(OERI) , the government seeks to develop quality 
teaching techniques and encourage school districts 
across the country to adopt the most successful of 
them. Certainly this is a worthy goal. There may 
not be a “magic bullet” for every problem in Amer- 
icas failing schools, but there are many proven, 
effective teaching methods that have been devel- 
oped and that could be replicated in the class- 
room. 

In light of OERIs potential to contribute signifi- 
cantly to the quality of the nations education, 
however, a review of its activities is especially dis- 
appointing. Its work is fragmented and apparently 
vulnerable to politicization and manipulation. 

This is especially true for OERIs regional educa- 
tion laboratories. These labs have been around for 

^ more than 32 years and have received over $750 

lo 
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o 
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million in federal funding. Yet, by nearly all 
accounts, what they have produced has been 
largely irrelevant to the class- 
room. Their work suffers 
from research overlap, an 
overemphasis on service 
delivery at the expense of 
quality, and a lack of objec- 
tivity and responsible scien- 
tific methodology. MoVeover, 
recent reports by the Depart- 
ment of Educations own 
inspector general have 
revealed significant misman- 
agement of federal funds at 
some labs. 
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A review of lab Internet 
sites turns up few details on 
free-market ideas or initia- 
tives such as school choice, 

even though some recent studies on school choice 
programs do demonstrate their successes and 
appeal to inner-city parents. On the other hand, 
the labs have been active promoters of fads. Rather 
than note proven methods of success to boost stu- 
dents’ acadernic achievement, they emphasize 
such measures as making students “feel positive” 
about the classroom and school environment, “sit- 
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uational learning,” “better academic self-concept,” 
and “developmentally appropriate” teaching. 

A June 1993 analysis of these labs by Maris 
Vinovskis, former Research Advisor to the Assis- 
tant Secretary at OERI and currently a professor at 
the Department of History and Institute for Social 
Research at the University of Michigan, provides 
further information on the questionable quality of 
research at five OERI regional education labs. 
Although Vinovskis praised the work of some labs, 
he concluded that after spending $811 million of 
taxpayer money between 1966 and 1991, the labs 
had little to show for it by way of success. His 
research found many flaws with the quality of 
research conducted as well as a disconnect 
between that research and practical advice for 
teachers. 

Federal education research labs, as currently 
funded and operated, do little if anything to pro- 
mote proven models of teaching that lead to 



higher academic outcomes. Lab funding should be 
tied to programs that boost academic outcomes. 
Moreover, either the labs should compete with 
other agencies and the private sector for federal 
grants to fund each project, or their funding 
should be turned over to the states to help individ- 
ual states set up their own research entities. 

Finally, policymakers should follow simple 
guidelines in deciding how to conduct and dis- 
seminate education research. The determining fac- 
tor in deciding whether a program will be funded 
and implemented in the schools should be 
whether that program aims to increase students* 
academic scores, not how much the educational 
bureaucracy claims it will boost childrens self- 
esteem or how much the education lobbyists like 
the program. 

— Nina H. Shokraii is Education Policy Analyst at 
The Heritage Foundation. 
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